Device for Acoustic Source Localization

ABSTRACT

Acoustic signals from an acoustic event are captured via sensing nodes of sensor group(s) that comprise a group of sensing nodes at a location comprising spatial boundaries. Each of the sensing nodes comprise a sensor area. Each of the sensor group(s) is based on: range limits of each of the sensing nodes; shared sensing areas of the sensing nodes; and intersections between the sensor area for each of the sensing nodes and the spatial boundaries. Solutions(s) are generated by processing the acoustic signals. The solution(s) indicate the location of the acoustic event. A strength of solution compliance value for at least one of the solution(s) is determined. A refined solution is generated employing: sensor contributions of sensing nodes; and the strength of solution compliance value with the spatial boundaries and at least one of the solution(s). A report is created comprising the location of the acoustic event.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.18/203,943, filed May 31, 2023, which is a continuation of U.S. patentapplication Ser. No. 17/685,761, filed Mar. 2, 2022, which is acontinuation of U.S. patent application Ser. No. 16/937,702, filed Jul.24, 2020, which is a continuation of U.S. patent application Ser. No.16/207,163, filed Dec. 2, 2018, now U.S. Pat. No. 10,746,839, which is acontinuation of U.S. patent application Ser. No. 15/873,917, filed Jan.18, 2018 now U.S. Pat. No. 10,180,487, which is a continuation of U.S.patent application Ser. No. 14/863,624, filed Sep. 24, 2015, now U.S.Pat. No. 9,910,128, which claims the benefit of U.S. ProvisionalApplication No. 62/138,474, filed Mar. 26, 2015, entitled “AcousticSource Localization in Confined Spaces,” which are all herebyincorporated by reference in their entirety.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a diagram of an example Sensor node according to aspects ofsome of the various embodiments.

FIG. 2 is an example block diagram showing components of a sensor nodeaccording to aspects of some of the various embodiments.

FIG. 3 is an example schematic illustration of sensors located in anindoor environment with special limits according to aspects of some ofthe various embodiments.

FIG. 4 is an example flow diagram showing a process of spatial limitingof sensor input according to aspects of some of the various embodiments.

FIG. 5 is an example sensor control screen according to aspects of someof the various embodiments.

FIG. 6 is an example shot sensor integrated into light according toaspects of some of the various embodiments.

FIG. 7 is an example diagram illustrating shot sensor and lightingcontrol mediation according to aspects of some of the variousembodiments.

FIG. 8 is an example diagram of a trajectory sensor according to aspectsof some of the various embodiments.

FIG. 9 is an example diagram of a muzzle velocity meter according toaspects of some of the various embodiments.

FIG. 10 is an example diagram of a trajectory sensor according toaspects of some of the various embodiments.

FIG. 11 illustrates an example of a suitable computing systemenvironment on which aspects of some embodiments may be implemented.

FIG. 12A is a table of time difference of example arrival values for acooperative cluster.

FIG. 12B is a table of example velocities of for a cooperative cluster.

FIG. 13 is an example flow diagram showing a process of spatial limitingof sensor input according to aspects of some of the various embodiments.

DETAILED DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention relate to the localization ofacoustic sources. This localization may apply to, for example, thelocalization of gunshots, explosives or other impulsive acousticsignals.

Embodiments of the present invention locate acoustic sources from eventsthat occur in defined spaces. One or more sensor nodes may be locatedwithin a confined area bounded by a physical structure or territorialboundary that also defines the set of possible source locations of theacoustic event. Source location detection incorporates the spatialboundary information with pre-determined sensor positions. Some of thevarious embodiments comprise command and control features wherein eachsensor is inherently registered with an acute space-time context.Command and control features may manage sensor contributions to thedetection and localization of a simultaneous event. A base station may:receive information from sensors; manage sensor groups and processsolutions. Some of the various embodiments may comprise command andcontrol features associated with surveying the sensor positions andtheir spatial environment, providing event triggers that actuate otherdevices or systems, and communications and messaging.

Acoustic localization as discussed herein relates to the problem ofgunshot detection and localization. Acoustic events such as gunshots maybe characterized by bullet muzzle blast and shockwave with relation tocaliber, weapon type, and other factors. Time difference of arrival maybe analyzed to localize these acoustic sources. Techniques to performlocalization may comprise time synchronization, signal classification,methods for filtering out erroneous data, and communicating with otherelements of a system that supports extended functionality such as acamera.

Some acoustic gunshot localization systems measure the muzzle blast orthe shockwave of the bullet, or both. A muzzle blast is an explosiveshock wave caused by a bullet being ejected from the barrel of a weapon.The muzzle blast may be emitted from the weapon and propagate inmultiple directions; however, the energy of the muzzle blast may besignificantly reduced in the opposite direction from where the bullet isfired. If a bullet is supersonic, the bullet may produce a shockwavethat propagates away from the projectile at the speed of soundperpendicularly to the direction of travel. The bullet shockwave mayhave a characteristic “N-Wave” form with a rapid time interval (e.g. 200μs) and the wave shape may be dependent on the caliber of theprojectile. A subsonic bullet, such as produced by many handguns, maynot form a bullet shockwave but only produce a muzzle blast. The muzzleblast signal may have a longer time interval (e.g. 2-3 ms) and may bedifficult to distinguish from other concussive sounds.

Single node systems in multi-path environments may have some performanceissues. A portion of existing acoustic gunshot detection systems use asingle sensor with an array of microphones designed for self-protection.Some single node systems may determine time difference of arrival of theshockwave signal captured at the microphones on the sensor to determinethe direction of travel of the projectile and therefore locate thedirection of the source. Distance to the acoustic source may beestimated by locating muzzle blast using a process that looks for thehighest energy signal that occurs sometime after the shockwave is found.Some single node systems may contain an array of microphones and employa neural network to analyze captured acoustic signals from an acousticevent to determine if the acoustic event may be classified as a gunshot.The neural network may comprise, for example, hardware configured as aneural network, and/or hardware in combination with software. If theacoustic event is classified as a gunshot, a further analysis may beemployed to look for the direction of arrival of the gunshot and use acamera to capture an image. In closed spaces or high multipathenvironments, there may be a possibility of reflections that can confusethe sensor as to the actual direction of the source. Highly energeticsignals such as a bullet shockwave passing in close proximity to thesensor or a standing wave created by a shockwave propagating in anenclosed space may overwhelm the acoustic microphone making it difficultto detect the entire signal. The possibility for a muzzle blast signalreaching the microphone at the same time as a bullet shockwave signal ora reflected shockwave signal may cause a mixing of signals making itdifficult to separate a shockwave from the muzzle blast signal.

Multi-sensor area systems may employ multiple sensor nodes placed aroundan area that allow for multiple simultaneous detections that may besynchronized to determine the source of an acoustic event (e.g. ashooter location). Area systems may capture acoustic detections from anarray of sensor nodes emplaced in different positions. For example,multiple nodes spaced apart may be employed to detect bullet shockwaveand muzzle blast signatures. The trajectory of the bullet may bedetermined by using information on the arrival times of a bulletshockwave detected at the various nodes and solving a ballistics model.Information obtained from the muzzle blast signal may be employed toestimate the range to the acoustic event (e.g. shooter). Acoustic sensornodes spaced apart may be employed to detect an event wherein at leastthree sensor nodes are configured and a common clock is employed todetermine the absolute time-of-arrival at each of the sensors. Thisinformation may be employed to triangulate on the location of the sourceof the acoustic event. The acoustic signal may be communicated to ahuman reviewer for verification that it was indeed a gunshot.

A plurality of spatially separated sensor nodes may be employed toobtain multiple detections of the same acoustic event. Sensor fusionmechanisms may be employed to identify possible source locations basedon a mechanism that favors results from multiple reporting nodes thatare most consistent. This process may, for example, employ a slidingtime window and count the maximum number of shot time estimates that arecalculated to be in that window. A viable solution may exist in thewindow with the maximum count. This mechanism may reduce multipatheffects in urban areas when: the direct line-of-sight (LOS) signal isthe highest energy signal; the multiple signals do not overlap; andthere is unambiguous time separation between direct LOS detections andreflected detections.

Indoor shot detection systems may be employed to detect acoustic eventsthat occur in indoor locations. These systems may be, in some cases,designed to be lower cost than the wide area systems and/or militarysystems. Gunshots may be detected using a simple assumption that agunshot has significantly higher signal strength (sound pressure level)in comparison to background noise and has rapid signal rise time. Indoordetection systems based on individual nodes may be confined to a room orarea and employ location information from where the sensor is within abuilding floor plan to identify a shot location. An audio signal may becommunicated to human reviewers to reduce the possibility of error.Optical sensors (e.g. one or more) may be employed in addition tomicrophones in order to optically verify the presence of a muzzle blastand reduce the false detection rate. Optical sensors may introduce falsedetections. These systems may or may not employ sensor fusion to removeambiguities.

False alarms may occur when there remains some ambiguity between a shotsignal and other concussive sounds. Another reason for false alarms isthat the strength of an acoustic signal may be dependent on the squareof the distance to the acoustic source. For example, a pistol fired tenor more meters from the sensor may have similar characteristics to otherimpulsive sounds like a locker slamming in close proximity to sensor.

Efforts to reduce false alarm rates that employ additional means ofverification that a gunshot occurred either with the use of orthogonalsensor inputs or human reviewers may be costly. If the means ofverification is a human reviewer, then as the number of installationsmay grow, the requirement for reviewers may increase. Use of an opticalsensor that looks for muzzle flash to verify that a shot was detectedmay increase overall reliability, but, in certain circumstances, such asnon-LOS conditions or when a flash suppressor is utilized, they may notdetect the flash and conditions such as bright sunlight may introducefalse alarms, therefore these sensors may not be expected eliminatefalse alarms. Due to the nature of the threat and the cost of respondingto false alarms, users of a gunshot detection system may have a lowtolerance for false detections and therefore may look for additionalsolutions for verifying shot reports.

Indoor environments may challenge single node systems. Some systemsdesigned for indoor environments employing single sensors in an areasuch as a classroom may employ a common assumption that the gunshotsignal is characterized by the presence of strong acoustic signal withrapid rise time. This assumption, by itself, may suffer from falsealarms, missed detections, errors caused by multipath, and impreciselocalization. A single node may miss detections due to failure to detectgunshots that are too distant to meet the threshold or are fired tooclose to the sensor and thus causing the microphones to saturate. Asingle node may mistake a concussive sound in very close proximity tothe sensor as a shot detection or confuse a bounced signal as a shotdetection or be unable to separate signals that are mixed withreverberations. Walls, ceilings and other features in interiorenvironments cause sound to be echoed and also give rise toreverberations inside the enclosure resulting in mixed signals that areambiguous. Localizing the source of individual shots may be limited tothe room the sensor is installed. For practical purposes, a singlesensor may not provide enough location information in areas withmultiple access points like a cafeteria or atrium.

Multi-sensor systems that combine detections from a plurality of sensorsmay overcome false alarms and problems that cause missed detections anderrors caused by multi-path and reverberations. Multi-sensor systems maycompute location-based energy peaks and implement search windows thatmitigate false alarms and reject multipath signals. A plurality ofsensors may be employed to determine accurate location and providetrajectory and caliber information that further confirm the presence ofa shot. These systems have been proven in outdoor environments, but theemployment of these systems indoors may benefit from enhancements asdescribed herein. Multi-sensor systems that rely on discrimination ofindividual acoustic events may be complicated in severe echo conditionswherein shockwave and muzzleblast signatures overlay. Some systems mayrequire clear LOS to the acoustic source from multiple sensors that aredistributed across a wide area outdoors. Indoors, the sensors in somesystems may likely be in linear arrays with very few having common LOSto the source. These systems may operate under the assumption that thesignal propagates uniformly across multiple sensors. This assumption maycause difficulties in interior spaces that are characterized by surfacesthat range from absorbent to reverberant, where the signal may varywidely across the array of locations, such as being mixed at one sensorand not mixed at another. Finally, the fusion process may requireaccurate position and heading information for sensor nodes. This may becomplicated indoors where GPS is not available.

Embodiments of the present invention addresses deficiencies with priormethods of acoustic localization in confined spaces. Some of the variousembodiments may sense and locate the source of an acoustic eventutilizing awareness of a sensor's spatial surroundings to limit thelocalization task. Some of the various embodiments may employpermissible sensing areas and decompose the problem of signal detectionand localization into multiple steps while applying constraints. One ofthe various factors comprises defining areas where each individualsensor can generate an acoustic source measurement with an open line ofsight. Another factor comprises defining the amount that variousindividual sensor measurements may contribute towards a fused solutionwithin a defined set of spatial constraints. Another factor comprisesdefining lists of spatial areas and their associated sets ofconstraints. Some of the various embodiments may manage the placement ofsensor node(s) to increase their contribution to performance. Geometricsensor areas may be located that limit the range at for which individualsensor node(s) may measure a signal and wherein cooperative sensingparameters for multiple sensors may be defined throughout an area, and acommon frame may be determined with regard to how various sensor(s)contribute towards a solution. A geometric area, such as a 2D or 3D box,circle, sphere, and/or the like may be determined surrounding the nodeposition. The geometric area may have deducted sectors with obstructedline-of-sight or that extend beyond other imposed spatial constraints.According to some of the various embodiments, cooperative sensingparameters may be determined as relating to the number of other sensornodes that share sensing areas. Node placement may be determined toincrease shared sensing areas and area coverage to reduce ambiguity.Solution results in qualified sensing areas may determine howinformation from designated sensing nodes are permitted to contribute toan overall solution.

Some of the various embodiments may be manifested as a gunshot detectionand/or bullet tracking system configured for use inside of buildingsand/or to cover limited outside areas such as, but not limited to,parking lots, campuses, firing ranges, compounds, combinations thereof,and/or the like. Some embodiments may be linked to other security andnotification infrastructure such as, but not limited to, an integratedsecurity system, a video management system, a cloud-based subscribernotification system, combinations thereof, and/or the like.

We now discuss locating the sensing areas in confined spaces. Thegeometric sensing area for each individual node may be determined byutilizing, at least in part, a map, a floor plan, other geo-referencedfeature boundary data, combinations thereof, and/or the like. Thislocation data may be used for indoor areas, areas confined by physicalwalls or structures, combinations thereof, and/or the like. The positionof a node may be located by, for example, using GPS and/or measuring theposition of the node relative to the floor plan or map. A relativereference frame may be employed if the floor plan and/or map are adigital file. In the case of a digital file, the scale may beestablished in the software file. The position of the sensor may beentered into the digital file. The sensor node's radial FOV up to amaximum distance may be specified. The areas where there are walls,obstructions, and/or other features that limit line-of-sight may bedetermined in the digital file. The sensor node's area may be comparedwith the areas of the floor plan that are obstructed from directline-of-sight from the sensor node. The obstructed areas may be deductedfrom sensor FOV creating the node's sensing areas.

We now discuss defining cooperative sensing parameters. For each node, alist of neighboring nodes may be determined as a set of nodes that arewithin the maximum distance from that node. This list of neighboringnodes may be determined at the set up. Up to a maximum number of nodesthat are specified may be included in a neighborhood. The sensing areasfrom neighboring nodes may be compared and the areas where there is anintersection of more than one neighbor listed. Multi-sensor fusion mayrequire a minimum number of participating nodes. Therefore, sensingareas where there is an intersection with the minimum number of nodesrequired for fusion may be determined. The minimum number could be, forexample, two nodes reporting for localization or three or more nodesreporting for trajectory information. One example may be the case of astraight corridor inside of a building. In this example, the sensors mayhave LOS in two directions and the possible source location may berestricted to a line. With the Time Difference of Arrival (TDOA)localization technique that is performed when the sensors are at knownpositions and the acoustic emitter is at an unknown position requires,three sensors may be needed to locate the source in two dimensions(x,y). If there is uncertainty in the system, then a fourth sensor maybe required to resolve the error in the position. In the narrow corridorexample where the source exists only along one line and the error isexpected to be very small, then only two sensors may be required tolocate the source. An embodiment of this invention may classify areasand number of sensors that can unambiguously determine the location ofthe source and designate those sensors as a contributing cluster. Whenan event occurs, a search window may look for the presence of detectionsfrom the minimum number of sensors in the cluster.

With regard to determining sensor placement. The placement of nodes maydetermine the amount of sensing area coverage and shared sensing areas.Sensors may be placed to take advantage of the desired sensing areawhile limiting the number of required sensor nodes.

Cooperative clusters may be determined. Nodes in the same neighborhoodthat share sensing areas may form a cooperative cluster. When anacoustic event takes place, each node that is exposed to the resultingsignal and makes detections may send a report across a network.Detection reports may be accumulated at a central node or gateway andlinked to reports from other nodes in the cooperative cluster.Cooperative clusters may determine the contribution from each sensor inthe fusion process. Fusion mechanisms may be employed to estimate thelocation of the source and other information such as trajectory from thereporting sensors in the cluster. Changing accepted contributions fromeach node in the cluster may change (or refine) the fused solution.

Some of the various embodiments sense and locate an acoustic sourceutilizing awareness of a multitude of sensor's spatial surroundings tolimit the localization task. Multiple acoustic sensing nodes may beemployed to detect impulsive acoustic events. Detected acoustic wavesmay be analyzed to determine whether the source of the events was agunshot and collaborate to produce a fused solution determining thelocation of the impulsive acoustic events (e.g. the location of ashooter).

FIG. 1 is a diagram of an example sensor node 100 according to aspectsof some of the various embodiments. Multi-channel acoustic sensing nodes100 may be employed. For example, as illustrated in FIG. 1 , four (4)microphones (e.g. 101A, 101B, 101C and 100D) separated by a minimumdistance of 10 cm may be employed in acoustic sensing node 100. A sensorhousing 102 may contain elements to detect and process acousticdetections at the microphones (e.g. 101A, 101B, 101C and 100D).According to some of the various embodiments, housing 102 may comprisean antenna 103 configured for wireless communications. According to someof the various embodiments, housing 102 may comprise an optical sensor104 configured to detect, among other things, explosive events.

FIG. 2 is an example block diagram showing components of a sensor nodeaccording to aspects of some of the various embodiments. Acoustic sensornode(s) 100 includes many components that may enable sensor node(s) 100to be configured for a range of applications. The acoustic environmentmay be continuously monitored by the microphone array (e.g. 101A, 101B,101C and 101C) and processed via analog channel(s) 240 and a processor220. Audio processors may collect the incoming signal in memory 221, andthe signal from the microphone(s) (e.g. 101A, 101B, 101C and 101C) maybe processed to measure features such as, for example, rise time andamplitude, compared against background noise for indications that theacoustic waves may be shockwave or muzzle blast signals. Measurementsfrom the microphone(s) (e.g. 101A, 101B, 101C and 101C) may be furthercompared to determine direction-of-arrival relative to the sensornode(s) (e.g. 100) orientation and length of the signal. Detections froman optical sensor 254 may be processed and/or employed to further verifyif a gunshot event occurred.

Multi-source fusion mechanisms may be employed. For example, whencharacteristic waves are detected, nodes may output data 210 to acentral host configured to execute a sensor fusion process to combinedetections from multiple nodes. Measurements from participating nodesmay be communicated through one or more of available communicationschannel 211,212,213,214,215, or 216, provided that the relative timeinformation among the cluster of nodes is adequately preserved. Aprocess may be executed wherein acoustic wave measurements from multiplesensors are used to locate the source and estimate the trajectory of asupersonic projectile. An example process, wherein the fusion process isbounded by a set of constraints derived from the surrounding spatialenvironment is described in the example flow chart shown in FIG. 4 .

Aspects of embodiments of the present invention may be employed tolocate sensors. Limiting the fusion process with constraints derivedfrom spatial surroundings 400 may comprise locating sensor positions andorientations in a reference frame relative to a node cluster and acoordinate system that may be used to collaborate with other nodes andshare results with outside subscribers. In one example embodiment, eachnode position may be measured with, for example, an on-board GPS 251 andits orientation and heading measured via accelerometers 252 and compassmodule 253. This information may be stored in a geocoded map. Manualinput may be employed for locating sensor positions in the absence ofGPS and the orientation and heading sensor suite. The sensor node'sradial Field-of-View (FOV) up to a maximum distance may be specified.The maximum distance may correspond to a maximum range from where agunshot will produce a detectable signal. Alternatively, an arbitrarycutoff distance may be employed.

Locating sensing areas may comprise determining the sensing areaboundaries for each node. Sensors may be placed in an area, and thesensor's spatial surroundings entered on a common vector map at 401. Thegeometric sensing area for each individual node may be determined byutilizing a map or floor plan as the input (See FIG. 3 ). This techniquemay be employed for indoor areas or areas confined by physical wallsand/or structures. The sensor node's maximum FOV may be specified. Theposition of each node (e.g. 301, 302, 303, 304, and 305) may be locatedrelative to the floor plan or map (e.g. 300). The position may beentered as, for example, as an x, y, z coordinate in a relativereference frame where, if the floor plan or map is a digital file, thescale is established in the software file. The areas where there arewalls, obstructions, and/or other features that limit line-of-sight(LOS) up to that maximum range may be determined in the digital file.The sensor node's entire area may be compared with the areas of thefloor plan that are obstructed from direct line-of-sight from the sensornode. The obstructed areas may be deducted from sensor FOV creatingpermitted sensing areas. From the perspective of implementation, if thesensor position is (x, y), and it's sensing area up to a maximum LOS iswithin an enclosed space, the sensor's sensing area may be restricted byR={(x,y)1−a:Sx,y:Sa}. In the example shown in FIG. 3 , the sensing areafor node 301 may be determined to be shaded area 308. Shaded area 308comprises shaded area 309. Similarly, the sensing area for node 304 maycomprise the union of shaded areas 308 and 307, and the sensing area fornode 305 may comprise the union of shaded areas 306 and 307. Anotherexample process for locating sensing areas may employ, for example, asighting tool such as a laser rangefinder to create a relative ofobstructive feature positions.

Sensing areas shared by the intersection of two or more nodes may beresolved at 402. The determination of the intersection of shared sensingareas may be performed from the set of neighboring nodes. A multi-sourcefusion algorithm may require a minimum number of sensors with shared LOSto the source. The minimum number could be two nodes reporting forlocalization and/or three or more nodes reporting for trajectoryinformation. Shared sensing areas (e.g. 309) may represent areas whereinthe fusion process may comprise good coverage and will most accuratelyresolve the position of the shooter when the fusion process takes placeat 403. Optimizing shared sensing may be performed through an iterativeoptimization process. The placement of nodes may be managed to achieve ahigh level of shared sensor coverage. An optimal deployment goal maymaximize shared space and minimize the required number of sensor nodes.

According to some of the various embodiments, nodes that share sensingareas may form a cooperative cluster at 404. When an acoustic eventtakes place, each node exposed to the resulting signal may makedetections and send a report across the cluster. Detection reports maybe accumulated at a central node. The reports may be submitted to afusion algorithm with weighting applied to groupings from the samecooperative cluster and an estimate of the location of the source andother information such as trajectory may be resolved. In another aspectof one of the various embodiments, the fusion process may be implementedto further refine the solution at 405 by comparing location andtrajectory solutions against a set of qualified sensing areas. Thefusion process may be adapted with weighting values associated withreports from contributing nodes that are consistent with sensing areasthat are in proximity of the estimated location and reduce the allowedcontributions for sensors where the estimated location is not part of asensing area.

Data collected from a live demonstration provides illustrates anembodiment of a computation within a cluster. Using the placement ofexample nodes in FIG. 3 , nodes 301, 302, 303, 304 may be in a sharedsensing area that is constrained in a narrow and long hallway. Sensorsmay be placed in a one-dimensional line (such as sensor nodes 301, 302,303, 304). The position of the nodes and subsequently, the distancebetween nodes may be known from the initial placement survey. When arifle shot is fired in the hall, the sensors may detect a range ofsignals depending on their location with respect to the source. Sensorsto the rear of the rifle may detect the muzzle blast signal and sensorsin the front may detect the shockwave coming from the bullet and amuzzle blast signal possibly at the same time. The time of arrival ofthe acoustic shot signal may be recorded at each of the sensors and thetime difference of arrival (TDOA) computed for each sensor pair in thecluster is shown in FIG. 12A. Sensors have known positions on the line,therefore the velocity of the acoustic signal travelling between sensorpairs is directly computed and shown in FIG. 12B. The data showsmeasurements between sensors 303 and 304 are consistent with the speedof sound indicating a muzzle blast signal. The measurements between 302and 301 are consistent with a rifle bullet and may represent a bulletshockwave. The other velocity measurements in the cluster are too largeto be practical indicating they are not measuring the same signal. Itmay be deduced that a shockwave signal travelled from sensor 302 towardssensor 301 and a muzzle blast only was present travelling from sensor303 to 304. The likely source location is between sensors 302 and 303with a supersonic shockwave signal propagating in the direction of 302(negative direction) and a muzzle blast propagating in the direction of303 (positive direction). The TDOA from 302 and 303 is found from FIG.12A to be −54.814 ms, noting the negative value indicates the signalarrived at 303 before arriving at 302. Using the known distance between303 and 302, and the propagation velocities for the bullet shockwave andmuzzle blast signal the location of the source may be found to be 20.83m from 302 in the direction of 303. This agrees with the sensing areaconstraint. All sensing clusters that contain measurements are similarlyevaluated, however, solutions yielding results that agree with the areaconstraints receive the highest weighting.

According to some of the various embodiments, hub reporting and sensorcontrols may be employed. A solution from the sensor cluster may becommunicated to a hub where it is reported to a data repository orintegrated with other services at 406. FIG. 5 is an example sensorcontrol screen according to aspects of some of the various embodiments.As illustrated in example FIG. 5 , a software interface to the hub mayallow for interactive control of sensor cluster(s). Reporting featuresillustrated in FIG. 5 show a post-shot scenario and shows sensor nodesthat reported a signal detection but were unable to calculate anyadditional information (nodes 501A, 501B, 501C, and 501D), nodes thatwere able to measure a muzzle blast signal 502, and a shockwave signal503, and a computed location within a shared sensing area 504. Thecontrols manage sensor placement and orientation may have been enteredvia manual input and/or a map as described above. Utility featuresincluded in the sensor setup controls 505 may manage individual sensorhealth and performance parameters. A calibration 506 utility may beconfigured for users to modify signal characteristics that may bemeasured and classified as an event. In another aspect of thisinvention, the remote access reporting and control features may beintegrated with other automated systems.

According to some of the various embodiments, acoustic source locationreports may be integrated directly into a video surveillance network.This configuration may require a map of camera locations and the area ofcoverage for each camera. The area covered by the acoustic sensornetwork may be linked to the camera coverage map. When an acousticsource is located, a report may be submitted that identifies thecamera(s) that have the specific location within their coverage. Thisreport may be employed to cue the camera(s) to begin streaming videoshowing the location of the acoustic source. The report may provideaccurate time data to enable synchronization of the video stream to theexact moment the acoustic event took place. This reporting mechanism mayemploy a broadband or other telecommunications link to create a videostreaming cue at a remote location such as a security operations center,an emergency dispatch center, a security services location, or a clouddata sharing location.

According to some of the various embodiments, the I/O port FIG. 2, 210 ,for at least one sensor nodes may receive one or more types of sensoractuation messages. For example, receipt of a sensor actuation messagemay cause the sensor to report the presence of an emergency situation atthat location. This may be referred to as a panic button message. Inanother example, receipt of a sensor actuation message may be employedto initiate a validation test of the health and functionality of thesensor node. This may be referred to as a system test message. Inanother example, receipt of an actuation message may instruct the sensornode to transmit a localization report to the network.

According to some of the various embodiments acoustic sensor node(s) maybe integrated into lighting fixture as shown in FIG. 6 . Thisconfiguration 600 refers to luminaires typically used for commerciallighting applications such as street lighting, parking lot lighting,parking garage lighting, exterior lighting and/or interior commercialspaces such as enclosed malls or shopping centers, and/or the like. Thisconfiguration may require the node to separate the microphones from theacoustic processor unit. Microphones 601 may be installed into theluminaire casing and the acoustic processor unit 602 housed in the boxthat normally contains the lighting control electronics. According tosome of the various embodiments, a wireless network may be employed (seeantenna 603) for sensor network communications.

According to some of the various embodiments, acoustic sensor nodemanagement and sensor lighting and control mediation may be employedinto cooperative sensor network for or an augmented lighting controlnetwork. The sensor network for the acoustic localization system mayoccupy the same physical medium. In the illustrated example embodimentshown in FIG. 7 the sensor network 700 may comprise a group of multipleintegrated sensor and lighting control systems (e.g. 710A, 710B, 710C .. . 710D). Multiple integrated sensor and lighting control systems (e.g.710A) may comprise integrated sensor and lighting control systems sensornetworks such as, for example: 750A, 750B . . . 750C. Sensor networkssuch as sensor networks (e.g. 710A, 710B, 710C . . . 710D) may occupy awireless Zigbee medium or other lighting control bus. The lightingcontrol electronics 702 may be configured to manage lighting sensors(701A, 701B, 701C . . . 701D) that comprise sensors configured to, forexample, check power levels, ambient lighting levels and/or the presenceof pedestrians or vehicles. Luminaire 703 may contain power and lightingactuation switches that provide power to the lamp as well as integratedelectronics. The acoustic sensor processor unit 602 may be integratedwith the lighting control electronics for the purposes of powermanagement and access to network communications. The lighting controlnetwork may be configured to tolerate latency and may have a muchdifferent architecture than the acoustic sensor network that may rely ona deterministic link architecture that may only intermittently be used.According to some of the various embodiments, a sensor networkchangeover module 704 configured to effectively toggle the networkparameters between the lighting and the acoustic sensor activities maybe employed. The network changeover module may activate middleware layer707 configured to control network activities of unit(s) in the network.When an acoustic sensor node reports a detection that requiresdeterministic use of the network, the middleware 707 may command achangeover 705 to the acoustic network at node(s) on the network. Afterthe last communication from the acoustic network has timed out, thelighting control electronics may resume scheduled communications and themiddleware 707 may arbitrate scheduling between lighting and acousticnode communications. Multiple sensor networks may be collected atgateways that route message traffic into either a lighting controlprocess 711 or a security alert process 712. The sensing and lightingcontrol system up to gateway 710A may occupy a portion of the lightingcontrol management or security management system. In a real-worldimplementation such as, but not limited to, an enclosed mall, parkinglot, and/or campus, there may be multiple integrated sensor and lightingcontrol systems (e.g. 710A, 710B, 710C . . . 710D) connected into, forexample, a single management application.

According to some of the various embodiments, an acoustic target may beemployed. FIG. 8 is an example diagram of a trajectory sensor accordingto aspects of some of the various embodiments. The acoustic target 800may comprise an electronic device configured to be utilized at rifleranges for accurately measuring target impact and communicating to thebench at the firing line, thereby reducing the need for individuals tomove down range and retrieve or change targets. In this exampleembodiment, acoustic channel(s) may process information about bulletshockwave detections and may be employed in combination to determine thelocation that the bullet intersects the microphone plane. In the exampleacoustic target, the microphones (801A, 801B, 801C . . . 801D) may bemounted into a frame 811 with a hollow center that may, according tosome embodiments, be over a meter in each dimension. The frame mayincorporate a mounting apparatus 812 that enables the target apparatus810 to be placed on a firing line and resilient enough to withstandoccasional bullet impact. The acoustic processor unit 820 may beseparated from the microphones (801A, 801B, 801C . . . 801D) and housedin a rugged casing 821 that may, for example, be buried or placed behinda berm with and connected to the target sensor apparatus 810 via a cable822 that extends up to several meters. The acoustic processor unit maycommunicate to the firing line via, for example, a wireless data link103.

According to some of the various embodiments, a muzzle velocity metermay be employed. FIG. 9 is an example diagram of a muzzle velocity meteraccording to aspects of some of the various embodiments. A muzzlevelocity meter 900 may comprise an apparatus employed to determine thevelocity of a bullet as it leaves a gun on the firing line 900. Thisembodiment may employ two (or more) electronic target apparatus 911 and912 as the basis. Electronic target apparatus 911 and 912 may be similarto electronic target apparatus 810. In this embodiment, two acoustictargets 911 and 912 may be employed with a smaller frame dimension (forexample, less than 1 meter). The targets 911 and 912 may be separatedapproximately a meter apart. An acoustic processor unit 920 (similar toacoustic processor 820) may be incorporated into the frame such that theapparatus is a single unit. The bullet may pass through both targetframes 911 and 912 so that the exact location and time that the bulletpenetrates each frame 911 and 912 is determined. This results in atrajectory and velocity determination.

According to some of the various embodiments, a trajectory sensor may beemployed. FIG. 10 is an example diagram of a trajectory sensor accordingto aspects of some of the various embodiments. The trajectory sensor1000 may comprise an apparatus employed to determine the trajectory of abullet as it passes downrange of a firing line. This embodiment may besimilar to the muzzle velocity meter 900 with frames 1011 and 1012oriented vertically such that the bullet passes above the apparatus.This embodiment may collect and process information employing acousticprocessor 1020 to obtain the bullet shockwave and to determine thevelocity and path of the bullet.

FIG. 11 illustrates an example of a suitable computing systemenvironment 1100 on which aspects of some embodiments may beimplemented. The computing system environment 1100 is only one exampleof a suitable computing environment and is not intended to suggest anylimitation as to the scope of use or functionality of the claimedsubject matter. Neither should the computing environment 1100 beinterpreted as having any dependency or requirement relating to any oneor combination of components illustrated in the exemplary operatingenvironment 1100.

Embodiments are operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well-known computing systems, environments, and/orconfigurations that may be suitable for use with various embodimentsinclude, but are not limited to, system on module (SOM), embeddedcomputing systems, personal computers, server computers, hand-held orlaptop devices, multiprocessor systems, microprocessor-based systems,set top boxes, programmable consumer electronics, network PCs,minicomputers, mainframe computers, cloud services, telephony systems,distributed computing environments that include any of the above systemsor devices, and the like.

Embodiments may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Someembodiments are designed to be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed computingenvironment, program modules are located in both local and remotecomputer storage media including memory storage devices.

With reference to FIG. 11 , an example system for implementing someembodiments includes a general-purpose computing device in the form of acomputer 1110. Components of computer 1110 may include, but are notlimited to, a processing unit 1120, a system memory 1130, and a systembus 1121 that couples various system components including the systemmemory to the processing unit 1120.

Computer 1110 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 1110 and includes both volatile and nonvolatile media, andremovable and non-removable media. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media includes both volatileand nonvolatile, and removable and non-removable media implemented inany method or technology for storage of information such as computerreadable instructions, data structures, program modules or other data.Computer storage media includes, but is not limited to, random accessmemory (RAM), read-only memory (ROM), electrically erasable programmableread-only memory (EEPROM), flash memory or other memory technology,compact disc read-only memory (CD-ROM), digital versatile disks (DVD) orother optical disk storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumwhich can be used to store the desired information and which can beaccessed by computer 1110. Communication media typically embodiescomputer readable instructions, data structures, program modules orother data in a modulated data signal such as a carrier wave or othertransport mechanism and includes any information delivery media. Theterm “modulated data signal” means a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia includes wired media such as a wired network or direct-wiredconnection, and wireless media such as acoustic, radio frequency (RF),infrared and other wireless media. Combinations of any of the aboveshould also be included within the scope of computer readable media.

The system memory 1130 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as ROM 1131 and RAM 1132. Abasic input/output system 1133 (BIOS), containing the basic routinesthat help to transfer information between elements within computer 1110,such as during start-up, is typically stored in ROM 1131. RAM 1132typically contains data and/or program modules that are immediatelyaccessible to and/or presently being operated on by processing unit1120. By way of example, and not limitation, FIG. 11 illustratesoperating system 1134, application programs 1135, other program modules1136, and program data 1137.

The computer 1110 may also include other removable/non-removablevolatile/nonvolatile computer storage media. By way of example only,FIG. 11 illustrates a hard disk drive 1141 that reads from or writes tonon-removable, nonvolatile magnetic media, a magnetic disk drive 1151that reads from or writes to a removable, nonvolatile magnetic disk1152, a flash drive reader 1157 that reads flash drive 1158, and anoptical disk drive 1155 that reads from or writes to a removable,nonvolatile optical disk 1156 such as a CD ROM or other optical media.Other removable/non-removable, volatile/nonvolatile computer storagemedia that can be used in the exemplary operating environment include,but are not limited to, magnetic tape cassettes, flash memory cards,digital versatile disks, digital video tape, solid state RAM, solidstate ROM, and the like. The hard disk drive 1141 is typically connectedto the system bus 1121 through a non-removable memory interface such asinterface 1140, and magnetic disk drive 1151 and optical disk drive 1155are typically connected to the system bus 1121 by a removable memoryinterface, such as interface 1150.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 11 provide storage of computer readableinstructions, data structures, program modules and other data for thecomputer 1110. In FIG. 11 , for example, hard disk drive 1141 isillustrated as storing operating system 1144, application programs 1145,program data 1147, and other program modules 1146. Additionally, forexample, non-volatile memory may include instructions to, for example,discover and configure IT device(s); the creation of device neutral userinterface command(s); combinations thereof, and/or the like.

A user may enter commands and information into the computer 1110 throughinput devices such as a keyboard 1162, a microphone 1163, a camera 1164,and a pointing device 1161, such as a mouse, trackball or touch pad.These and other input devices are often connected to the processing unit1120 through a peripheral input interface 1160 that is coupled to thesystem bus but may be connected by other interface and bus structures,such as a parallel port, game port or a universal serial bus (USB).Additional devices may be connected to peripheral input interface 1160,such as for example, acoustic processor 1165 configured to detect bulletshockwave information. A monitor 1191 or other type of display devicemay also connect to the system bus 1121 via an interface, such as avideo interface 1190. Other devices, such as, for example, speakers 1197and printer 1196 may be connected to the system via peripheral interface1195.

The computer 1110 is operated in a networked environment using logicalconnections to one or more remote computers, such as a remote computer1180. The remote computer 1180 may be a personal computer, a hand-helddevice, a server, a router, a network PC, a peer device or other commonnetwork node, and typically includes many or all of the elementsdescribed above relative to the computer 1110. The logical connectionsdepicted in FIG. 11 include a local area network (LAN) 1171 and a widearea network (WAN) 1173 but may also include other networks. Suchnetworking environments are commonplace in offices, enterprise-widecomputer networks, intranets and the Internet.

When used in a LAN networking environment, the computer 1110 isconnected to the LAN 1171 through a network interface or adapter 1170.When used in a WAN networking environment, the computer 1110 typicallyincludes a modem 1172 or other means for establishing communicationsover the WAN 1173, such as the Internet. The modem 1172, which may beinternal or external, may be connected to the system bus 1121 via theperipheral input interface 1160, or other appropriate mechanism. Themodem 1172 may be wired or wireless. Examples of wireless devices maycomprise but are limited to Wi-Fi and Bluetooth. In a networkedenvironment, program modules depicted relative to the computer 1110, orportions thereof, may be stored in the remote memory storage device. Byway of example, and not limitation, FIG. 11 illustrates remoteapplication programs 1185 as residing on remote computer 1180. It willbe appreciated that the network connections shown are exemplary andother means of establishing a communications link between the computersmay be used. Additionally, for example, LAN 1171 and WAN 1173 mayprovide a network interface to communicate with other distributedinfrastructure management device(s); with IT device(s); with usersremotely accessing the Peripheral Input Interface 1160; combinationsthereof, and/or the like.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example, and notlimitation. It will be apparent to persons skilled in the relevantart(s) that various changes in form and detail can be made thereinwithout departing from the spirit and scope. In fact, after reading theabove description, it will be apparent to one skilled in the relevantart(s) how to implement alternative embodiments. Thus, the presentembodiments should not be limited by any of the above-describedexemplary embodiments.

In addition, it should be understood that the figures and algorithms,which highlight the functionality and advantages of the presentinvention, are presented for example purposes only. The architecture ofthe present invention is sufficiently flexible and configurable, suchthat it may be utilized in ways other than shown in the accompanyingfigures and algorithms. For example, the steps listed in any flowchartmay be re-ordered or only optionally used in some embodiments.

It should be noted the terms “including” and “comprising” should beinterpreted as meaning “including, but not limited to”.

In this specification, “a” and “an” and similar phrases are to beinterpreted as “at least one” and “one or more.” References to “the,”“said,” and similar phrases should be interpreted as “the at least one”,“said at least one”, etc. References to “an” embodiment in thisdisclosure are not necessarily to the same embodiment.

It is the applicant's intent that only claims that include the expresslanguage “means for” or “step for” be interpreted under 35 U.S.C. 112.Claims that do not expressly include the phrase “means for” or “stepfor” are not to be interpreted under 35 U.S.C. 112.

The disclosure of this patent document incorporates material which issubject to copyright protection. The copyright owner has no objection tothe facsimile reproduction by anyone of the patent document or thepatent disclosure, as it appears in the Patent and Trademark Officepatent file or records, for the limited purposes required by law, butotherwise reserves all copyright rights whatsoever.

Further, the purpose of the Abstract of the Disclosure is to enable theU.S. Patent and Trademark Office and the public generally, andespecially the scientists, engineers and practitioners in the art whoare not familiar with patent or legal terms or phraseology, to determinequickly from a cursory inspection the nature and essence of thetechnical disclosure of the application. The Abstract of the Disclosureis not intended to be limiting as to the scope in any way.

1. A device for locating a source of an acoustic event comprising: oneor more processors; and memory storing instructions that, when executedby the one or more processors, cause the device to: capture acousticsignals from an acoustic event via sensing nodes of at least one sensorgroup, the at least one sensor group comprising a group of sensing nodesat a location, wherein: the location comprises spatial boundaries; eachof the sensing nodes comprise a sensor area; and each of the at leastone sensor group based on: range limits of each of the sensing nodes;shared sensing areas of the sensing nodes; and intersections between thesensor area for each of the sensing nodes and the spatial boundaries;and generate at least one solution by processing the acoustic signals;determine a strength of solution compliance value for at least one ofthe at least one solution; generate a refined solution employing: sensorcontributions of sensing nodes; and the strength of solution compliancevalue with the spatial boundaries; determine a location of an impulsiveacoustic signals based at least in part on at least one of the at leastone solution and the refined solution; and a hub configured for controlof the group of sensing nodes at the location and for communicating areport including the location of the acoustic event to one or more of adata repository and a service at a remote location via a communicationslink.
 2. The device of claim 1, where the one solution and the refinedsolution are utilized to discriminate false alarms due at least in partby multi-path acoustic waves.
 3. The device of claim 1, where the onesolution and the refined solution are utilized to discriminate falsealarms due at least in part to reverberations.
 4. The device of claim 1,where the one solution and the refined solution are utilized todiscriminate false alarms due at least in part to signalcharacteristics.
 5. The device of claim 1, further comprisinginstructions that cause the device to determine a false alarm based atleast in part on the group of sensing nodes.
 6. The device of claim 1,further comprising instructions that cause the device to generate thereport indicating the type of acoustic event based at least in part onthe acoustic signals.
 7. The device of claim 1, where the locationcomprises an enclosed space.
 8. The device of claim 1, where theenclosed space is indoors.
 9. The device of claim 1, where the group ofsensing nodes comprises multi-sensors.
 10. The device of claim 1, wherethe device is operatively coupled to one or more interfaces.
 11. Thedevice of claim 1, where the impulsive acoustic signals comprise one ormore types of noises.
 12. The device of claim 11, where the types ofnoises include gun shots, muzzle blasts, bullet shockwaves, explosivesand other noises.
 13. The device of claim 1, where the impulsiveacoustic signals are categorized.
 14. The device of claim 1, where thedevice is operatively coupled to one or more networks.
 15. The device ofclaim 1, where the location comprises shared enclosed space and outdoorspace.
 16. The device of claim 1, further comprising the hub configuredto receive the report, the hub comprising at least one of the following:a communications hub; a sensor control gateway; and the data repository.17. The device of claim 1, further comprising the hub configured toextract incoming audio data collected in the memory and to process saidaudio data to further verify if the acoustic event occurred.
 18. Thedevice of claim 1, further comprising the hub configured to collectaudio data from the group of sensing nodes at the location thatcontinuously monitor the acoustic environment of the location.
 19. Thedevice of claim 1, further comprising the hub configured to send areport when the acoustic event occurs that includes collected audio dataor processed audio data.
 20. The device of claim 1, further comprisingone or more of the sensing nodes of the group of sensing nodes manuallyaligned to a mounting location of the one or more sensing nodes tolocate the sensor orientation of the one or more sensing nodes relativeto the mounting location.
 21. The device of claim 1, further comprisingone or more of the sensing nodes of the group of sensing nodes installedin a luminaire, where the one or more of the sensing nodes installed inthe luminaire are aligned with respect to the luminaire.
 22. The deviceof claim 1, where the location of the group of sensing nodes providesthe sensor position of the group of sensing nodes relative to a map ormap image and a geocoded sensor location and orientation of each sensingnode of the group of sensing nodes that can be stored.
 23. The device ofclaim 1, where the hub is configured to compute and map a field of viewof a sensing node of the group of sensing nodes and where a maximumdistance of the radial field of view corresponds to a maximum range fromwhere an acoustic event will produce a detectable signal.
 24. The deviceof claim 23, further comprising the hub configured to correlate thefield of view of the detectable signal with the locations and area ofcoverage of sensing nodes of the group of sensing nodes nearby thesensing node.